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Abstract 

^ | ' In this paper, we consider the nonasymptotic sequential estimation of means of random 

. variables bounded in between zero and one. We have rigorously demonstrated that, in order 

! to guarantee prescribed relative precision and confidence level, it suffices to continue sampling 

until the sample sum is no less than a certain bound and then take the average of samples as an 
estimate for the mean of the bounded random variable. We have developed an explicit formula 
and a bisection search method for the determination of such bound of sample sum, without any 
, knowledge of the bounded variable. Moreover, we have derived bounds for the distribution of 

sample size. In the special case of Bernoulli random variables, we have established analytical 
| and numerical methods to further reduce the bound of sample sum and thus improve the 

efficiency of sampling. Furthermore, the fallacy of existing results are detected and analyzed. 
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1 Introduction 

In various fields of sciences and engineering, it is a frequent problem to estimate the means 
of bounded random variables. Specially, Bernoulli random variables constitute an extremely 
important class of bounded variables, since the universal problem of estimating the probability of 
an event can be formulated as the estimation of the mean of a Bernoulli variable. For examples, 
the problems of estimating network reliability [9], the probability of acceptable performance of 
uncertain systems [18] [22] and approximating probabilistic inference in Bayesian network [7] can 
be cast into the framework of estimating the means of Bernoulli variables. 

Clearly, Bernoulli variables can be considered as a special class of random variable bounded 
in [0, 1]. In many applications, one needs to estimate a quantity /U which can be bounded in [0, 1] 
after proper operations of scaling and translation. A typical approach is to design an experiment 
that produces a random variable X distributed in [0, 1] with expectation fi, run the experiment 
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independently a number of times, and use the average of the outcomes as the estimate [6]. This 
technique, referred to as Monte Carlo method, has been applied to tackle a wide range of difficult 
problems. For instances, estimating multidimensional integration, volume and counts [9] |20j, 
finding approximate solution to enumeration problems [T7], approximating the permanent of 0-1 
valued matrices [IB] , solving the Ising model of statistical mechanics |15] , evaluating the bit error 
rate of communication systems [19] . 

Since the estimator of the mean of X is obtained from finite samples of X and is thus of 
random nature, for the estimator to be useful, it is necessary to ensure with a sufficiently high 

confidence that the estimation error is within certain margin. The well known Chernoff-Hoeffding 

In 2 

bound [5] |14j asserts that if the sample size is fixed and is greater than then, with probability 
at least 1 — 5, the sample mean approximates fx with absolute error e. Often, however, fi is small 
and a good absolute error estimate of n is typically a poor relative error approximation of \x 
[BJ. Therefore, we seek an (e,5) approximation for \x in the sense that the relative error of the 
estimator is within a margin of relative error e with probability at least 1 — 5. Since the mean 
value \x is exactly what we want to estimate, it is usually not easy to obtain reasonably tight lower 
bound for fi. For a sampling scheme with fixed sample size, a loose lower bound of /j, can lead to 
a very conservative sample size. For the most difficult and important case that no positive lower 
bound of \i is available, it is not possible to guarantee prescribed relative precision and confidence 
level by a sampling scheme with a fixed sample size. This forces us to look at sampling methods 
with random sample sizes. 

The estimation techniques based on sampling schemes without fixed sample sizes have formed a 
rich branch of modern statistics under the heading of sequential estimation. Wald provided a brief 
introduction to this area in his seminal book [23] . Ghosh et al. offered a comprehensive exposition 
in [10]. In particular, Nadas proposed in [21] a sequential sampling scheme for estimating mean 
values with relative precision. Nadas's sequential method requires no specific information on 
the mean value to be estimated. However, his sampling scheme is of asymptotic nature. The 
confidence requirement is guaranteed only as the margin of relative error e tends to 0, which 
implies that the actual sample size has to be infinity. This drawback severely circumvents the 
application of his sampling scheme. Due to the inherent unknown statistical error, asymptotical 
methods have been criticized in some literatures (see, e.g., [9], [13], and the references therein). 
Especially, researchers in the areas of randomized algorithms, controls and communication systems 
are very reluctant to use asymptotic methods for quantifying the uncertainty of estimation for 
purpose of avoiding another level of uncertainty, namely, the unknown error of inference (see, e.g., 
[18] . [19] , [20] . [22] and the references therein). Nevertheless, when nonasymptotic method is not 
available or too conservative, one has to resort to asymptotic methods. 

In recent years, aimed at making Monte Carlo estimation a more efficient and rigorous method, 
Dagum et al. and Cheng have attempted to develop nonasymptotic sequential methods for esti- 
mating means of random variables bounded in [0, 1]. To guarantee prescribed relative precision 
and confidence level, Dagum et al. proposed in [BJ that one should continue sampling until the 
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sample sum is no less than a threshold value. Obviously, this is simply a generalization of the 
classical inverse binomial sampling pj] [12j . However, the determination of the threshold of sam- 
ple sum is not trivial. Dagum et al. provided an explicit formula for computing such threshold 
value for ensuring prescribed relative precision and confidence level. In [3], Cheng attempted to 
improve the efficiency by using a smaller threshold value. 

In this paper, we revisit the sequential estimation of means of random variables bounded in 
[0, 1]. We discovered that Dagum et al. and Cheng have left major flaws in the determination of 
threshold of sample sum. Specifically, the proof of Dagum et al. for their claim on the reliability 
of estimator is incomplete and the gap cannot be filled by using their arguments. The proof 
of Cheng for his claim on the reliability of estimator is basically incorrect. Most importantly, 
we have developed a new approach to determine the smallest value of threshold and thus make 
the sampling much more efficient. An explicit formula for the threshold of sample sum is also 
derived, which is substantially smaller than that of Dagum et al. A direct consequence of our 
explicit formula is that Dagum's claim can be proved as a special result of ours. Moreover, we 
have derived general bounds on the distribution of sample sizes. Our method applies to arbitrary 
random variables bounded in [0, 1]. In the special case of Bernoulli random variables, we have 
developed a method to further reduce the threshold value and thus improve the efficiency of 
sampling. In particular, a computational method is established for computing the minimum 
threshold value when knowledge of the Bernoulli parameter is available. 

The remainder of the paper is organized as follows. In Section 2, our general theory of 
inverse sampling is presented. We discuss inverse binomial sampling in Section 3. In Section 4, 
we illustrate an application example in the performance evaluation of communication systems. 
Section 5 is the conclusion. All proofs are given in the Appendices. The mistakes of existing 
works are examined in Appendices D and E. 

Throughout this paper, we shall use the following notations. The expectation of a random 
variable is denoted by E[.]. The set of integers is denoted by Z. The ceiling function and floor 
function are denoted respectively by [".] and |_-J (i-e., \x] represents the smallest integer no less 
than x; [x\ represents the largest integer no greater than x). The left limit as t tends to is 
denoted as lining. The notation means "if and only if". The other notations will be made 

clear as we proceed. 

2 General Inverse Sampling 

Let X be a bounded random variable defined in a probability space (O, J^~, Pr) such that < X < 1 
and M[X] = fj, £ (0, 1). We wish to estimate the mean of X by using a sequence of i.i.d. random 
samples X±, X2, ■ ■ ■ of X based on the following inverse sampling scheme: 

Continue sampling until the sample size reach a number n such that the sample sum Y2T=i -^i 
is no less than a positive number 7. 
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We call this an inverse sampling scheme, since it reduces to the classical inverse binomial 
sampling scheme |12| in the special case that X is a Bernoulli random variable. 
We shall consider the following two estimators for 

7-1 



7 
n 



n 



1 



Specially, when X is a Bernoulli random variable and 7 is an integer, Jl and Jl are, respectively, 
the maximum likelihood estimator and the minimum variance unbiased estimator for the binomial 
parameter [8] [11] [12]. It should be noted that Jl is not an unbiased estimator of the binomial 
parameter; the bias may be considerable for small values of 7. 

To control the uncertainty of estimation, for a margin of relative error e € (0, 1) and a confi- 
dence coefficient 5 £ (0, 1), it is highly desirable to determine minimum 7 such that Pr{|/I — a\ < 
eu} > 1 — 6 when the estimator Jl is used, and Pr{|/i — u\ < e^i} > 1 — 5 when the estimator Jl is 
used. 

For this purpose, we have 



Theorem 1 Let e £ (0, 1) and 7 > 1. Let X\, X2, ■ ■ ■ be a sequence of i.i.d. random variables 
defined in a probability space (ri,J^",Pr) such that < Xi < 1 and E[Xj] = fi £ (0)1) f or 
any positive integer i. Define Jl = — and Jl = ^ry, where n is a random variable such that 
n(uj) = min {n G Z : Y^i=i Xi(ui) > 7} for any u> 6 Q. Define 

l + e N 



£(e,*y) = (l+e)- 7 exp 



and 



&{e,i) = (l + e)- 7 exp 
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1 + e 

£7 
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exp 
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Then, the following statements hold true. 



(I) Pr \ 

(II) Pr 



> e| < =S(e, 7) provided that 7 > 
> e| < 7) provided that 7 > ^ 



l-£ 



(TiT) J2(e,j) is monotone decreasing with respect to 7 > ^+^. Moreover, for any 5 E (0,1), 
i/iere exists a unique number 7 > suc/i i/mi J2(e,j) = 5. 

(IV) J2?(e,7) is monotone decreasing with respect to 7 > ~. Moreover, for any 5 S (0, 1), i/iere 
exists a unique number 7 > - s-uc/t i/ia£ J2(£, 7) = 5. 



f v? (i - £)7 < 7 < 7 < (14 iwi^)- e 



lira 

5^0 



7 



ln(l + e) - 



1+e 



(1+e) In j 4(e-2)(l+ e )lnf M 
^ (21n2-l)e 2 ^ P " moTeolJ(lT > 



lim 

e^0 



ln(l 



e - 



1+e 



(1) 



(vj; For e > m 



Pr < n > 



7(1 + e) 



(i+e-^x^ 



(l+«-^-M)X; 
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(VII) For < q < 1 - fi, 



< (1 - Q) 



(l-e)x^ 



1-M 

1- Q- (1 



(l-e-M)xJ 



See Appendix A for a proof. From Theorem[TJ we can see that 7 and 7 can be readily computed 
by a bisection search method by making use of the monotone properties and the bounds provided 
in (III), (IV) and (V). 

As an immediate application of Theorem [H we can easily determine the bound (i.e., threshold 
value) of sample sum without a lower bound of fi. Specially, we have 

Corollary 1 Let e, 5 & (0,1) and 7 > 1. Let X\, X2, ■■■ be a sequence of i.i.d. random 
variables defined in a probability space (f2, J^,Pr) such that < Xi < 1 and E[Aj] = \l G (0, 1) 
for any positive integer i. Define Jl = 3- and = ^Ej, where n is a random variable such that 
n(w) = min {n & Z : Y^7=i ^ 7} f or an V w € 0. Then, 



Pr 



fM-fJ, 



< e } > 1 - 5, 



Pr 



ix- n 



/' 



<e^ > 1-6 



provided that 



7 > 



;i + e)ln| 



(1 + e) ln(l + e) -e 



(2) 



(3) 



Corollary [T] provides an explicit bound of sample sum in the inverse sampling scheme to ensure 
the reliability requirements ([2]). Actually, as can be seen from Theorem HJ an implicit bound, 7, 
makes the sample scheme more efficient while guaranteeing ([2]). When e or 5 is small, the explicit 
bound is close to the implicit bound, as indicated by (H|). 

In [6], Dagum et al. claimed that, in order to ensure 



Pr 



< e } > 1 - 5, 



(4) 



it suffices to have 7 greater than Ti = 1 + 4 ( e 2 H 1+£ ) \ n |_ p or same purpose of guaranteeing 



flU), Cheng claimed in [3] that 7 can be reduced as a 
equation 



(l+£)ln 
(l+e)ln(l+e)-£' 



where 5, satisfies the 



(1 - 6.) + 



l + e 
r+2i 



l+e 
l + 3e 



6. 



However, as can be seen from our analysis in Appendices D and E, their arguments in justification 
of the claims are fundamentally flawed. 

The chain of inequalities of statement (V) of Theorem [T] show that our explicit bound @ 
is significantly smaller than Ti. This indicates that the bound Ti, obtained by Dagum et al., 
indeed suffices the need of ensuring (j3|), though their proof is not correct. 
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Although Cheng failed to prove his claim on the reliability of fi, he obtained in [5j the following 
useful bounds on the average sample size: 

^ < E[nl < 1 + 1 (5) 

by making use of the observation that X\ + • • • + X n ^\ < 7 < X\ + • • • + X n and Wald's identity 
to conclude that /i(E[n] — 1) < 7 < /dE[n] and thus ©. 

From ([5]), it can be seen that the average sample size is almost proportional to 7. Hence, it 
is reasonable to compare the efficiency of different inverse sampling schemes by their bounds of 
sample sum 7. For this purpose, we have plotted our explicit bound, implicit bound 7 and the 
bound, Ti, of Dagum et al. in Figs. [THU It can be seen that the bound of sample sum of Dagum 
et al. is too conservative and leads to a substantial waste of sampling effort. 




Margin of Relative Error e 

Figure 1: Bounds of Sample Sum versus Margin of Relative Error (5 = 0.05) 



3 Inverse Binomial Sampling 

For the special case that X is a Bernoulli random variable, the sampling can be made more 
efficient. When no knowledge of the binomial parameter is available, we have the following results 
that can be used to determine the threshold value, which is smaller than its counterpart in the 
general inverse sampling. 

Theorem 2 Let e G (0,1). Let X\, X2, ■■■ be a sequence of i.i.d. Bernoulli random variables 
defined in a probability space (fl, J^, Pr) such that Pr{Xj = 1} = p G (0, 1) and Pr{Xi = 0} = 
l—p = q for any positive integer i. Define p = 3-, where 7 is a positive integer and n is a random 
variable such that n(uo) = min{n 6 Z : J27=i = 7} for any u G f2. Then, Pr | 2z£ > e| < 




Figure 2: Bounds of Sample Sum versus Margin of Relative Error (5 = 




Figure 3: Bounds of Sample Sum versus Margin of Relative Error (5 = 
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Margin of Relative Error e 



Figure 4: Bounds of Sample Sum versus Margin of Relative Error (5 = 0.05) 

J2(e, 7) where £?(e,j) = (l+e)~ 7 exp ( -^^j +(l-e) -7 cxp (— jz^J; which is monotone decreasing with 



respect to 7. Moreover, for any 5 G (0, 1), there exists a unique number 7* such that J2(e,j* 
and maxj ^^J^ , n _\,J n _j^ F \ < 7* < 7 < TTTTTTTTTTTTw ■ Furthermore, 



Pr< n > 



)-e' (l-e)ln(l-E)+ 

7(1 + g) 



p 



< 



| < 7" < 7 < 
l-p 



1-P+Q 



(l+e)ln(l+e)-e ' 
(l-p+fi)xj 

+ 



(i+e)x: 



and 



Pr 4 n < 



7(1 -f?) 



< 



l-p- 



(i-p-e)x ; 



(i-e) 



(i-e)xj 



e > o 



< Q < 1 - p. 



See Appendix B for a proof. From Theorem [21 it is clear that 7* can be readily obtained by 
a bisection search. 

When the binomial parameter p is known to be bounded in [a, b] C (0,1), it is desirable to 
further reduce the conservativeness by a computational method. For instance, one may wish to 
determine the smallest 7 such that Pr| 2z£ < e| > 1 — 5 for any p G [a, b] C (0, 1). For this 
purpose, an essential computational routine is to check whether a given value of 7 is large enough 
to ensure Pr||^£ < s\ > 1 — 5 for any p G [a,b]. At the first glance, it seems necessary to 
evaluate Pr j 2z£ < s j for infinite many values of p. Fortunately, our following result indicates 
that the number of evaluations can be reduced to finite. 

Theorem 3 Let e G (0, 1). Let X\, X2, ■ ■ ■ be a sequence of i.i.d. Bernoulli random variables 
defined in a probability space (0, J^, Pr) such that Pr{Aj = 1} = p G (0, 1) and Pr{Xj = 0} = 
1 — p = q for any positive integer i. Define p = ^ and p = ^5y, where 7 is a positive integer and 
n is a random variable such that n(u) = min{n e Z : Y%=i = 7} f or an D ^ G ^- Then, the 
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minimum o/Pr 



p-p 

v 



< el with respect to p € [a, b] C (0, 1) is achieved on the set 



{a,6}u|^ 



7-1 



7-1 
(l + e)(* + 7 -l) 



G (a, 6) | I = 0, 1, • • • , oo 
€ (a, 6) \£ = 0,1,--- ,oo}. 



Similarly, the minimum of Pr 
set 



p-p 
p 



< ej with respect to p £ [a, b] C (0, 1) is achieved on the 



{a, b} U 



7 



(!-£)(* + 7) 



U 



7 



(l + e)(£ + 7 ) 



G (a,6) |^ = 0,1, ■•■ ,oo 



G (a, 6) | £ = 0, 1, • ■ • ,oc 



See Appendix C for a proof. The application of Theorem [3J in the computation of minimum 
7 is obvious. For a fixed value of 7, since the minimum of coverage probability with respect to 
p £ [a, b] is attained at a finite set, it can determined by a computer whether 7 is large enough to 
ensure Pr j 1 < £ j > 1 — <5 for any p € [a, 6]. Starting from 7 = 2, one can find the minimum 7 
by gradually incrementing 7 and checking whether 7 is large enough. 
For p = p e = n4 . F 7 f^-i) G (°» & )> we have 



(l+e)(<?+7-l) 
p-p 



Pr 



P 



< £ 



Pr 



7-1 



(1 + e)p 
Pr < n -7 < £ 

Z+\ 1^+7-1)1-1 



1 < n < 



7-1 
(1 - e)p 



E 



2e 

7 + «-l 



(i + 7-1) 



For p = pi 



'-1 



(1-6)^+7-1) 

p-p 



jy E (a, 6), we have 



Pr 



P 



< £ 



Pr 



Pr U 



7-1 
(l + e)p 

2£ 



1 < n < 



7-1 



l + £ 



(1 - e)p 
+ 7-l)<n-7<£ 



i=max(0, «-[^(<+7-l)]+l) 



Convenient formulas for the computation of Pr | 2z£ < e | can be derived in a similar way. 

We would like to note that the method of reducing the number of evaluations of coverage 
probability can also be developed for the problems of computing minimum fixed sample sizes 
for the estimation of Poisson parameter, proportions of infinite and finite populations. In this 
direction, we have recent research works [I] [2] [5J. 
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4 An Application Example 



In this section, we shall illustrate the application of the general inverse sampling method in 
information technology. Consider the evaluation of bit error rate performance of a communication 
system. The stream of bits are divided as blocks of bits with length L > 1. Each block is 
modulated as waveforms and transmitted via a noisy channel. At the receiver side, the block 
of bits are recovered by demodulation. Due to the impact of noise, there may be incorrectly 
recovered bits. Let Z be the number of erroneous bits. Then, ^ is a random variable bounded in 
[0, 1], assuming possible values ■£, £ = 0,1, ■ ■ ■ ,L. The bit error rate can be defined as 



P, = E 



Since each block of bits are modulated and demodulated identically and independently, we have 
a sequence of i.i.d. random variables Z\, Z%, ■ ■ ■ , which have the same distribution as Z. To 
estimate P e , we can continue the simulation of the modulation and demodulation process until 
the number of blocks reach a number n such that 

L(l + £ )ln| 



> (l + e)hi(l+e 



An estimate of the bit error rate can be taken as 

- 1 



n 



1 



l+£)lnf 



(1 + e) ln(l + e)-e 



Then, by our explicit formula ([3]), 



Pr 



Pr. - P. 



Pr 



<£> > 1 — 5. 



It should be noted that existing asymptotic estimation methods are not appropriate in this context, 
since the bit error rate P e is usually very small. Special results for Bernoulli random variables are 
not applicable since j- is a random variable assumes (L + 1) > 2 values. 



5 Concluding Remarks 

The problem of finding relative precision estimates for means of random variables bounded in 
between zero and one has numerous applications and has been studied in history for a long period 
of time. Despite the lack of rigorous justification, it was a significant progress made by a number 
of researchers in realizing that the sample mean ensures the prescribed reliability once the sample 
sum reaches a certain threshold value. Our main contributions are two folds. First, we have 
discovered critical mistakes exist in the determination of the threshold value, which determines 
the reliability of the estimate and the efficiency of sampling. Second, we have developed explicit 
formulas and computational methods to calculate the threshold value, which make the sampling 
as efficient as possible, while guaranteeing prescribed relative precision and confidence level. 
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A Proof of Theorem 1 



Lemma 1 Let e G (0,1) and 7 > ^. Let 77 = ^ry- anc ^ Q be a number determined by = 
+ i. T/ien, 0<r?<C<e<l. 

Proof. Since < e < 1 and 7 > i, we have 7 > 1 and < r/ = e — < £ < 1- 

To show < C < 1) it suffices to exclude three possibilities. First, ( / 1 because e / 1. 
Second, £ > 1 is impossible, otherwise < ^ 7 < 1 — e, contradicting to 7 > ~. Third, 
(" < is impossible, otherwise < 1 + - 7 < contradicting to 7 > ~. 
Finally, since < e < 1, < C < 1 and jrf = 7 ~1^ £ > 1, we have C > r /- 

□ 



We need to use some inequalities on the function (p(x) = ln(l + x) — for |x| < 1. 

e' 2 ^ e 2 1 \ ^ £ 2 (21n2- 

(fci) > T > > 1+F 



Lemma 2 lei e G (0, 1). T/ien, p(-e) > ^f^y > f > 99(e) > £ ^| 1} > 



Proof. To show 99(e) < it suffices to note that ip(e) = \ = for e = and that 



p^jp- - e < for < e < 1. 

2 2 

To show </?(— e) > 2(i- g ) ) it suffices to show (1 — e) ln(l - e) + e — > for e G (0, 1). This 
is true because the left side assumes value for e = and its derivative is — ln(l — e) — e > for 
any e G (0, 1). 

Define /(e) = (1 + e)ln(l + e) - e - (2 In 2 - l)e 2 . To show 99(e) > it suffices to 

show /(e) > 0. Note that /'(e) = fn(l + e) - 2(2 In 2 - l)e and /"(e) = ^ - 2(2 In 2 - 1) > 
if e < 2(9in2-i) ~~ Hence, /'(e) is increasing for < e < 2 (2in2-i) ~~ ^ anc ^ decreasing for 
2(2 i n 1 2 _ 1) - 1 < e < 1. Since f(O) = and f(l) = ln2 - 2(21n2 - 1) = 2 - 31n2 < 0, we have 

that there exists a unique null point e* G ( 2(21112-1) ~~ ^' ^) °^ /'( e ) - This implies that /(e) is 
monotone increasing for < e < e* and monotone decreasing for e* < e < 1. Observing that 
/(0) = /(l) = 0, we can conclude that /(e) > for any e G (0, 1). □ 

In - _i 

Lemma 3 Let e, 5 G (0, 1) and 7 > -^A. Let rj = ^J.. Then, ip(-rj) > 99(e) > 0. 



Proof. 



Since 99(0) = and 99' (e) = 1 it follows that 99(e) > for any e G (0, 1). Note that 

l- v = l-(e-?—^)=(l-e)- ' 



7 — 1 / 7 — 1 

and 

V £ - — £ 1 



l- ?? (l-e)^r 1-e (l-e)7 
<p(-rj) = In 



7 

Hence, 



(l-e)(l + 



7-1 



e 1 

+ 



1-e (l-e) 7 
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In 4 



for 



By the third inequality of Lemma El we have ip(e) < %- for e G (0, 1) and thus 7 > 
any 5 € (0,1). Since — j > 0, using the inequality ln(l + x) > with x = — j, we have 



m 1 + ^ 



7-1 



- and thus 

7 



1 



<p(-ri) > ln(l - e) + - + - 

7 1-e (l-e)7 



ln(l - e) + 



1 



Define w(e) = ln(l — e) 



— - (i- E) in4 - Appling 7 > we have -(ir^ ^ - (i- £ )i n4 
<p(—T}) > w(e). To show 99(e) < (p(—rj), it suffices to show 99(e) < w{e). Note that 99(0) — w(0) = 
and 



(l-e) 7 

^■3 



> 



and thus 



y'{e)-w'{e) = 



(1 + e) 5 



3e 2 



(1-e) 2 (1-e) In 4 (1-e) 2 In 4 



e 2 [u(e) -8 In 2] 



(l + e) 2 (l-e) 2 ln4 

where u(e) = (1 + e) 2 (3 — 2e). Since u'(e) = 2(1 + e)(2 — 3e) = if e = §, the maximum of 
w(e) over interval [0, 1] must achieve at e = 0, | or 1. It can be checked that u(0) = 3, u(l) = 4 
and u (|) = ^ < 8 In 2. This shows that u{e) < 8 In 2, which implies (p'(e) — w'(e) < for any 
e € (0, 1). It follows that < ip(e) < w(e) < ip(—rj). 

□ 



A classical result due to p3] is restated as Lemma [Has follows. 

Lemma 4 Define Jt{z,n) = In (£) + (± - l) In (1E7) /or < z < 1 and < < 1. Let 
Xi,--- ,X n be i.i.d. random variables bounded in [0,1] with common mean value /i G (0,1). 
T/ien, Pr|S^i > z | < cxp (nz^#(z, ^)) for 1 > z > n = E[JQ]. Similarly, Pr | SLi < z | < 
exp {nzJi (z, fx)) for < z < /j, = E[Xj] . 



Similarly, 



Lemma 5 ^((1 + e)fi,fi) is monotone decreasing with respect to e € ^0, i — 1 
^#((1 — e)fi,(i) is monotone decreasing with respect to e € (0, 1). 

Proof. For e G ^0, i — 1^ , we have < (1 + e)/i < 1 and 

+ £)(!, (i) 

1 



<9e 



9s 



In 



1 



1 



1 



Ml 



In 



- 1 In 



1 — u 



1 + e u(l+s) 2 \l-n(l + e) 
1 . ( I -fx 



1-M 
l-//(l+e) 

1 



Ml + e) 



l-//(l + e) 



In 



l-/x(l + e) 



Similarly, 
lemma. 



9i 



M(l-e) 



■hi 



< 0. 



< for < e < 1. This completes the proof of the 

□ 



12 



Lemma 6 .#((1 + e)/i,/i) is no greater than —<p(e) for < fi < yjp. Similarly, -#((1 — e)/x,/i) 
is no greater than —<p(—e) for < /i < 1. 



Proof. First, note that 



lim ^#((1 + e:)//, 



lim 
In 
In 

In 
In 



1 + e 



+ 



1 



lim In 



1 In 



l-ix 



l-n(l + e) 



+ lim 



1 



In 



1 + e J ^0 \l-ij(l + e)J ^o/j,(l + e) \1 - /x(l + e 



l-/i 



\ In(l-/i)-hi[l-Ai(l + e)] 



1 + e At^o 



+ lim 



M(l+£) 

1 , 1+e 
1-M ^ l-A»(l+e) 



l + £ / M^O 
1 

+ 



1 + e 

Me) 



1 + e J 1 + e 

and, similarly, lim ([1 _o ~4K{(1 — = l n (l^i) — T^F = — v( — £ )- Next, we need to show that 

./#((1 + e)fj,,fi) is monotone decreasing with respect to //. To this end, note that 
cL#((l + e)u, a) 



d/j, 



d_ 

d_ 

da 



In 



1 



1 + e 
1 



1 



1 In 



1 — u 



u{l + e) 



u(l + e) 'J \l-(x(l + e) 
1 — u 



1 In 



1 - m(1 + e) 



u 2 (l + e) \l-fi(l+e)J \fi(l + e) 



M 2 (H 




1 








1 




M 2 (H 




1 




M 2 (H 





ln 



In 



1 — [i 



1 



1 — u 



M(l + e) 
1 



1 



1 



-1 - 



1-u l-u(l + e) 
1 \ 1 



ln 



1 — fj, 



1-u J u 
1 1 



u 2 (l+e) \1- fi(l + s)J n(l- fi)(l + e) 1-u u 



In 



1 — u 



u 2 (l+e) \l- f i(l + e)J fi(l-u){l + e) 



< 



if In f , l=JL , ) > i.e. 

l-ju(l+e) y — 1-M ' 



ln[l-^U< £/i 



(6) 

i-nj- 1-H y J 

Since < /i < j^, we have < < 1. Using the fact that ln(l — x) < —x for any x G (0, 1), 
we can conclude © and thus establish the monotone decreasing property of ^#((1 + e)fi,fi). 
Similarly, to show that ^#((1 — e)fj,,fj,) is monotone decreasing with respect to /i, note that 



cL#((l - s)u,u) 
du 



d_ 



ln 



1 



1 



1 — e J \u(l — e 
1 In' ^ 



- 1 In 



l-/x 



e 



< 
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V 1 — A 4 / 1 — M 

Since > 0, using the fact that ln(l +x) < x for any x 6 (0, oo), we can conclude (J7]) and thus 
establish the monotone decreasing property of ^#((1 — e)fj,, fi). 

Finally, since both functions ^#((l + e)/i, /i) and ^#((1 — /i) are monotone decreasing with 
respect to /i, these two functions must be bounded from above by their corresponding limit values 
as \x tends to 0, which have been obtained at the beginning of proof. This proves the lemma. 

□ 



Lemma 7 M f;^jq?^j^j, £*j is monotone decreasing with respect to g > 



Proof. Let 



7(i+e)-A* 

d[j^(z,n)} 
dg 



and m = 2. For q > ^, we have < z < ji, ^ > and 



Idz 



z dg~^ 



1 



-J-—\ n 

z 2 dg 

m dz 



z dg 



In 



z 

1-2 

1 - z 



1 <9z 



1 <9z 
— — — — In 



1 — z dg z 2 dg 



1-M 
1 - z 



7717 ^ Z 

mz 2 dg 



In 



<9m 
9g 



In 



1-M 

1 - z 



1-M 

1 - z 

<0, 



which implies that /i) is monotone decreasing with respect to g > ^. This proves the lemma. 

□ 



We are now in position to prove Theorem 1. 



A.l Proof of (I) 



By the definition of the estimator fj, = 



Pr 



>e)-=Prin< 



M(l + e) 



Pr < n > 



/i(l-e) 



Hence, we shall derive upper bounds for the tail probabilities Pr jn < } an d Pr jn > m^z^ }• 

We first bound Pr jn < ^ +e ^ } ■ Since n is an integer, we have 



Pr<n< 



7 



Pr < n < 



7 


}^ Pl { 


_/i(l + e)_ 





//(!+£*) 



where e* is a number such that ,,7 »n 



t^-t . Clearly, e* = 7 T , -1 > e > 0. For simplicity 



of notation, let m 
If m = 0, then 



12 — Since m is a nonnegative integer, it can be zero or a natural number. 



M(l+e*) 



Pr ^ n < 



Pr{n < m} = < exp(— ^ip{e)). 
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Otherwise if m > 1, then 



Pr < n < 



Pr{n <m} = Pr{X x H + X m > 7} = Pr{X > z}, 



cases. 



Ai(l+£* 

where X = — - and 2 = 3- = ail + e*) > a. Now we shall consider three 

(i) : In the case of z > 1, we have Pr{X > z} < Pr{J2"L 1 ^ > m l = < cx p( — 7V( e ))- 

(ii) : In the case of z = 1, we have pi = 14 L £ , , m = 7 and 

Pr{X > z] = Pr J £ X t = m I = [] Pr{JQ = !}<[] E [ X ^ = ^ 



. i=l 



1 + e* 



< 



7 



< exp(-7(p(e)). 



(iii) : In the case of fj, < z < 1 , by Lemma HI we have 

Pr{X > z} < exp(mz^(z, ^)) = exp(-y^ ((1 + e*)fi, /i)). 

Since e* > e, it must be true that /i(l + e) < + e*) < 1 and that J%{(\ + £*)fj,,fi) < 
.^#((1 + e)fi,fi) as a result of Lemma[5j Hence, 



Pr ^ n < 



7 



Pr{X >z}< exp(7„#((l + e)/x, //)). 



M(l + e) 

By LemmaEJ we have ^#((1 + e)//,/x) < — (/?(e). It follows that 



Pr ^ n < 



/!(!+£) 



Pr{X > z} < exp(-7¥>(e)). 



(8) 



Therefore, we have shown 



Prln< 



1 



< exp(-79j(e)) 



M(l + e) 
for all cases. 

We now bound Pr |n > j^rzi) }• Since n is an integer, we have 



Pr ^ n > 



M(l-e) 



Pr ^ n > 



M (l- £ ) 



Pr < n > 



1 



M(l-e) 



(9) 



Let C be a number such that — ^ = + -. Then, < C < e as a result of Lemma HJ By the 
definition of we have _ 1 > ^ fr_i — ^ j = f° r an y A* £ (0> !)■ Hence, 



Pr < n > 



7 

At(l-e) 



< Pr < n > 



7 



Pr < n > 



15 



with C* satisfying 
positive integer and 



2 



M(i-C) 



. Clearly, 1 0. Let m = Then, m is a 



Pr{n > m} = Pr{Xi + • • • + X m < 7} = Pr{X < z} 
where X = ^ i=1 Xl and z = (1 — C*)/- 4 - Applying Lemma [U we have 



Pr ^ n > 



Mi - 



= Pr{X < z} < exp(mz^£'(z, /x)) 
= exp(7-#((l-C*)/i,M)). 
Note that ^#((1 — C*)m> m) < -#((1 — C)/^ A 4 ) as a result of 1 > (* > ( > and Lemma[5j Hence, 

Pr{X < z} < exp( 7 ^((l - C)m,aO)- 



By Lemma[6l we have ^#((1 — < — ^(— £). It follows that 



Pr <^ n > 



7 



< exp(-7(/?(-C)). 



Thus, we have bounds for the two tail probabilities as follows: 



(10) 



Pr ^ n < 



M(l + e) 



< exp(— 7(p(e)), Pr < n > 



M(l-e) 



< exp(-7v?(-C)). 



It follows that 



Pr 



/' 



> e 



Pr < n < 



1 



+ Pr<n> 



fi(l + e)J { -/i(l-e) 

< exp(-7<^(e)) +exp(-7^(-C)) = =2(e,7) 



where we have used the definitions of £ and ip(.) in the last equality. This completes the proof of 
statement (I). 



A.2 Proof of (II) 

By the definition of the estimator /x = we have 



Pr 



/' 



t~" — 1 + Pr i n > 1 + 



To bound Pr jn < 1 + }' we snan consider two cases. 

(i) : In the case of (1 + e)fi > 1, we have Pr jn < 1 + jf+^j;} < Pr{n < 7} = < Pr jn < 

because n > J^ILi -^i — 7 ^ s always true. 

(ii) : In the case of (1 + e)fi < 1, we have Pr jn < 1 + (f+^l} - Pr ( n - (i+e)M }• 
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Therefore, in both cases, we have Pr jn < 1 + (x + ^ } < Pi' | n < (i+e)^ } an d 5 D Y virtue of 

7-1 



Pr < n < 1 



(l + e)/i 



< exp(-7V3(e)). 



(11) 



To bound Pr |n > 1 + (fr^j}, let 7? = ^Er and note that 

7-1 



Pr i n > 1 



(l-e)/i 



= Pr jn > 1 
= Pr < n > 



7-1 
(1-eV 
7-1 



Pr < n > 



(1 - e)n 

7 



(1 - 

< exp(-7^(-?7)) 



(12) 



where (|12p follows from a similar method as proving (jlOp . 

Combining (jlip . (| 12|) and invoking the definitions of 77 and </?(.) yields Pr||^^ > e| < 
cxp(— 7¥>(e)) + cxp(— 7V?(-?/)) = J?(e, 7). This completes the proof of statement (II). 



A.3 Proof of (III) 

Note that 7) = exp(— 7</?(e)) + exp(— 7</?(— £)), where C is determined by = + K By the 
chain rule of differentiation, we have 



3[^(£,7)] 
<?7 



-tp(e) exp(-7<^(e)) - exp(-7^(-C)) 



^(-0+7 



M-0 9C 
dC c»7 



< 



by observing that tp(e) > 0, <p(-() > 0, > and |^ > 0. This proves that B{e,i) is 

monotone decreasing with respect to 7 > ^Er- 

The existence and uniqueness of 7 in interval (-^,00) can be seen by the monotone decreasing 
property of J2(e, 7) with respect to 7 > — - and the fact that 



lim J2(e, 7) = 0, lim =S(e, 7) > lim 

7^00 ^_ » 7^ — 



7 - 1 +j 

7 - £7 



cxp 



1 — £7 — e 
l-e 



A.4 Proof of (IV) 

Note that =S(e, 7) = exp(-7<y9(e)) + exp(— 7<^(— 77)), where 77 = ^fr- By the chain rule of differentia- 
tion, we have 



d[£(e,*y)] 
c?7 



-<p(e) exp(-7</j(e)) - exp(-7p(-rj)) 



<y3(-7?) +7 



dip(-rj) dr/ 



dq (?7 



< 



by observing that (/9(e) > 0, <p(—ij) > 0, ^ > and j£ > 0. This proves that =S(e, 7) is 
monotone decreasing with respect to 7 > |. 



9") 
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The existence and uniqueness of 7 in interval (-,00) can be seen by the monotone decreasing 
property of J2(e, 7) with respect to 7 > ^ and the fact that 

lim B(e,i) = 0, lim M(e,i) > lim ( 7 - 1 ^ exp f^—^l] = 1. 

7-00 7 _i 7 ^iy 7 - £7 y \1-Sj 



A.5 Proof of (V) 



In 2 

First, we shall show the upper bounds for 7. For this purpose, note that, for 7 > ^7^, we have 

ln- r 



By the third inequality of Lemma [2j we have ^Ar > > ~. Since 7) = 5 and =S(e, 7) is 



< (^(e) < <p(—r]) as a result of Lemma [3l Hence, =S(e, 7) < 2 exp(— 7^(e)) < <5 for 7 > 

In § 2 In " 

monotone decreasing with respect to 7 > ~, it must be true that 7 < ^7^- Applying Lemma [21 

we have S? <r ln I <r (1+e)ln j <r 4(e-2)(i+ e ) lnj 
we nave 7 < ^7 < (2i„2-i) e 2 < P ■ 

Second, we shall show 7 < 7. Clearly, if 7 < -, then 7 > 7 is trivially true since 7 > -. Thus, 
we can focus on the case that both 7 and 7 are greater than K For 7 > -, by Lemma [TJ we have 
i] < ( and consequently J2(E, r y) < =S(e, 7). As a result, <!> = =S(e, 7) = =S(e, 7) > =S(e, 7). Since 
J(e,7) > =S(£,7) and 3[ ^' 7)1 < 0, we have 7 < 7. 

Third, we shall show (1 — e) 7 < 7. In light of the fact that 

£7-1 1 1 1 

V = — — T' 1 = 1 7 + -' 

7 — 1 1 — e 1 — C 7 

we have 

1-17 = (i-e)^, i-C = J ^, ^ = ^ 

V 7-1 l- 1 ^ 1-C 7" 

Therefore, if (1 — e)j > 7, then > 1, which implies £ < V- As a result, 

exp (-7^(e)) + exp (-tV(-C)) > exp (-7>0)) + exp (-J(p(-T])) 

> exp (-7VO)) + exp (-7X-77)) = S, 

which contradicts exp (— j(p(e)) + exp (— Jtp(— C)) = ^( £ il) = <5- Hence, it must be true that 
(1 -e)7 < 7. 

~ ~ ~ Ini ~ 

Now, we shall show ([T]). Since ip(e) > and exp(— 7<^(e)) < 5 = =S(e, 7), we have < 7. 
Combining this lower bound with the previously established upper bound yields 

In 7 „ In I 

< 7 < — (13) 



p(e) y?(e) 

Clearly, as an immediate consequence of (fT3|) , we have lim^o [ y ( e )]-i i n 2 = 1 • ^ remains to 
show lim e ^o , , m-i , = 1- To this end, we need to prove lim e _o cx p(~^( £ ))/^ = 1. It suffices to 
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show lim e _>o lYp{ e ) ~ <^( — C)] = 0- By virtue of (fT3j) and the condition = + i, 



i-e 1 



£7 



7 



> 



i-g y(s) 



1-e 



(14) 



Making use of the inequality of (|14p and the facts that 
have 

1 _ l-e y( £ ) 

limmt — > hm 

e^O £ e-»0 l 



l-e 



and 7 > ^—^ — > oo as e — > 0, we 



On the other hand, limsup e ^ | < 1 because C < e. Hence, 



lim — = 1. 

£^0 £ 



Applying the upper bound in (fl~3l) , we have 



(15) 



S 1 1 1 

exp (-7VOO) > - = -=S(e, 7) = - exp (-7>(e)) + ^ ex P (-tV(-C)) 

which leads to exp (— 7<^(e)) > exp (— 7<^(— C))i or equivalently, 

p(e) - tp(-C) < 0. 

By (USD, (USD and (El), we have 



(16) 



lim 

£^0 



y(g) - y(-C) 



lim 

£^0 



1 



V(-C) fCY e 



C 2 W 



2 ,-2 



1 - lim 

£—0 ( 2 

1 

1 - - x 1 x 2 = 
2 



2 

x lim 1 — 1 x lim 

£^0 V e J £^0 ip(£) 



and consequently, 



In I 

lim sup 7[<p(e) - y(~C)] < lim sup -t4- [<p(e) - = 0, 

£^0 £ — ¥>(e) 



In 2 

liminf7[^(e)-^(-C)] > liminf -^(e) - ¥>(-C)] = 0. 



It follows that lim e ^o7k(e) - ¥>(~C)] = and thus lim E ^ _°*? ( ~^ (e ^, = 



Finally, since e*p(-7y(0) + °x P (-7-y(-0) = 1 and lim ^ j^p 



exp(-7v(-C))/<5 



i , which implies lim £ ^o -t-^t 
1 ln 5 



(-jy V (e))/5 _ 
exp(-7^(-C))/<5 



1, we have lim £ ^ 



cxp(~7y(e)) 



1. 
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A.6 Proof of (VI) and (VII) 

First, we shall derive the upper bound of Pr jn > 7 ^ g ^ j. Since n is an integer, we have 



Y>i{n> 



7(1 + 0) 



/' 



where g* is a number satisfying 



Piin> 



7(1 + 0) 



Piin> 



7(1 + 0*) 
A* 



7(l + g") 



7(l+g) 



7(l+g") 



1. Since g > ^, we have 



70+g) 



Clearly, g* > g by the definition of g*. Let 
> 1, which implies m > 1. Hence, 



Pr ^ n > 



7(1 + 0) 



with X = 2 -'= 1 ' and z = ■ 

m 

follows from Lemma H] that 



Pr{n > m + 1} = Pr{Xi + • • • + X rn < 7} = Pr{X < z] 
3- — 7A1 _ , Note that < z < /i as a result of £>*7 > £7 > fi. It 



Pr < n > 



7(1 + g) 



Pr{X < z} < exp(mz^(z, //)) 



exp 



Since > g > ^, applying Lemma [71 we have 



Pr n > 



7 

7(1 + g) 



7(1 + g*) - n 

7 (l+e*)-/,^l 



— — , pi ) and 



7(i+e)-Ai" 



< cxp [7^ 

< cxp I 7^# 



7// 



7(1 + A*' 

7M 



cxp 7 



7(1 + g) — fi 

fj, 



fj, 



In 1 + 5 



1 



i±^-i-l|ln 



7 



A 4 7 



1 - 



7(i+e)-/i 

(l+«-?-M)Ki 



for g > -. 

7 



Now we bound Pr jn < 7 ^ g ^ |. Invoking (JSj) , we have 

< exp(7^#(^(l + e),/i)) 
In 



Pi ^ n < 



7 



M (l + e) 

= exp ^7 

for /i(l + e) < 1. Letting £> = we have 

7(1 -q) 



1 



1 



l + ej \n(l+e 



- 1 In 



1 - M (l + e) 



Pr<n< 



/' 



< exp 7 



7 

exp I - 



In(l-e) 



1-g 



1 In 



1-A* 



1 - 



(1 - g) In (1 - £>) + (1 - g - n) In 

(i-«-f)x; 



1-M 

1-Q-H 



1-M 

1- Q-fi 



for < < 1 — /x. 
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B Proof of Theorem 2 



We need the following preliminary result. 

Lemma 8 Let k = n - 7. Then, Pr{fc > s} < (^q) S (^p) 7 / or s > E[k]. Similarly, 
Pr{fc <s}< (^q) S (^p) 7 V < s < E[k]. 

Proof. Note that fc is a negative binomial random variable with distribution 



For any t > 0, 



Pr{fc = A;} 



Pr{fc > s} 



7 + k ~ 1 )p'rJ 
k 



k = 0,1,2, 



p r { e t(fc-«) > 1} < E 

OO 

E 



,t(fc-s) 



- ts E [e* fc ] 



k=0 



fc=0 



It can be checked that 



riln<ft(t) 
dt 



eT j . ,u uau uc uici^ mat — jg— - - a -r jr^r - if a = (s + 7 )ge*, 
> because s > E[k] = Substituting ge* = and e~ ts = (^-q) 3 



where (j>(t) = e 
in which t = In i 

in 0(t) yields the upper bound of Pr{fc > s}. Similarly, the upper bound of Pr{fc < s} can be 
established for < s < E[k]. 

□ 

Now we are in position to prove Theorem 2. By the definition of the estimator p — k\-^ > 



Pr 

To bound Pr jfc 



P-P 



P 

7 



> e > = Pr ik < 



1 



7 \ + Pr <^ k > 



p(l + e) 'J ' " I""- p(l-e) 
|, we need to consider three cases as follows. 

(i) : In the case of p{l + e) > 1, we have Pr jfc < p ^ +e ^ — 7j = < cxp (— ^ytp(e)). 

(ii) : In the case of p(l + e) = 1, we have Prjfc < - 7 | = Pr{fe = 0} = = (1 + e)-' 1 < 

exp (-7</?(e)). 



(iii): In the case of0<p(l + e)<l, applying Lemma [8] with s = ^q^j 
have 



7 < J -7 = E[fe], we 



Pr{k< 



p(l+e) 



7f < 



s + 7 
J 

7 



TTTF)- 7 / 7 \ 7 



_2 



p(l+e) T 



7 



l-p(l + e), 
exp {-yjt((l + e)p,p)) 



l + e 
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By the first statement of Lemma [61 we have ^#((1 + e)p,p) < —ip{e). 
Therefore, Pr jfc < p( ^ +e ^ — 7j < cxp (— 7V?(e)) is true for all cases. 

To bound Pr jfc > ^-3^ — 7}, applying Lemma [8] with s = p(xL e ) — 7 > ^ — 7 > E[fc] , we have 



7 /s+7\/s+7 



Vx{k> —- r - T > < 



p(l-e) J V s 

9 



l-p(l-e), 
= exp(7^((l-e)p,p)). 

By the second statement of Lemma [6j we have ^#((1 — e)p,p) < —<p(—e) and thus Pr{fc > 
2 7} < cxp (— 7</?(— e)). Combining the two bounds of the tail distribution probabilities of k, 

Pr 



p(l-e) 

we have 

P-P 



> e \ < exp (-7v?(e)) + exp (-7¥>(-e)) = 7)- 
P J 

Clearly, =S(e, 7) is monotone decreasing with respect to 7. Because of such monotone property 
and the fact that lim 7 ^ 00 J2(e, 7) = 0, lim 7 ^o 7) > 1 ; there exists a unique number 7* such 
that <=S(e,7*) = 6. 

To derive the lower bound for 7*, applying Lemma @ we have < (p(e) < <p(—e) and thus 
2 exp (— 7v?(— e)) < =2(e,7). It follows that 

max{2exp (-7V(-e)) , exp(-7>(e))} < 5 = =0(e,7*), 

from which we can obtain the lower bound of 7*. 
To derive the upper bound for 7*, note that 

<S(e, 7) - J2(e, 7) = exp (-j<p(-e)) - exp (~7<p(-C)) < 



because < ( < e < 1 and > for < £ < 1. Hence, 5 = Jg(e,7*) = ^(£,7) > *2(e,7)- 

~ In 2 

Since 7) is monotone decreasing with respect to 7, it must be true that 7* < 7 < - 



Finally, we consider the distribution of sample size. Note that 

Pr jn > 7(1 + g) j=Pr{fc> S } 
where s = (1 — p+f?)x^. Note that ^7^9 = gx pr~f~ an d ^-pP = 1 + Q- By Lemma El 
Pr jn > lQ±£l \ = p r {fc>s} 

/S + 7V/S + 7'" 

(i + e)' 



2 

¥>(e)" 



1 - p + 0, 

(i- P +e)x2 

P (l + ^+(l-P+f)X: 



1 — p + Q 

1 — p 
1 — p+ Q 



1 „ \(l-P+e)x£ 
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Changing the sign of g to negative yields 



P J \1 — p — Q 

This completes the proof of Theorem 2. 



C Proof of Theorem 3 

For simplicity of notations, define C(p) = Pr {|p — p\ < ep} and £(7, g, h,p) = J2i= g ( 7+ j" 1 )p 7 (1 — pY 
Then, 



C(p) 



Pr 



7-1 



fc + 7 - 1 



„ f 7-I , 7 — 1 

Pr / - 7 + K k < r -7 + 1 

Pr{^(p) < fe < %)} = S(j,g(p),h(p),p) 



where 



s(p) 



7-1 



7 + 2, h(p) 



7-1 ' 
;i - e)p 



It should be noted that C(p), g(p) and are actually multivariate functions of p, e and 7. 
We need some preliminary results. 

Lemma 9 Let pe = ( 1 _ £ ) 7 (7+ 7 „i) where £ € Z. T/ien, = h(pi + i) = £ for any p G (pi+i,P£)- 
Proof. Note that 



r 7-1 1 




(1 - e)p 


-7 = 



7-1 p/ 



(1 - e)pi p 

Since 1< ^ < ^ = ^|±2_ for p E (p^+i,^), we have 

,P£ 



7 



+ 7-1 



P 



7- 



£+7-1 < 



(£ + 7-1)- 



P 



< 



+ 7-1) 



£ + 7 
+ 7-1 



+ 7- 



Hence, £ — 1 < h(p) < £. Since is an integer, it must be true that h(p) = £ = h(pe + i) 



□ 



Lemma 10 Let p e = (1+e wi+ 1} where £ G Z. T/ien, = #(^) = £+1 for any p £ (pt+i,m)- 



(l+e)(M- 7 -l) 
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Proof. Note that 



g(p) 



7-1 
(l + e)p. 



7 + 2 



7 ~ 1 Pi 
(1 +e)p^ p 



7 + 2 



(£ + 7-1) 



P 



7 + 2. 



Since 1 < ^ < ^- = ^it^ for p G (p^+i,^), we have 



+ 7-1 < 



+ 7 - 1) — 
P 



< (£ + 7-1)- 



+ 7 



£ + 7- 



+ 7-1 

Hence, £ + 1 < g(p) < £ + 2. Since g{p) is an integer, it must be true that g{p) = t + 1 = g(pe). 



□ 



Lemma 11 Lei a < (3 be two consecutive elements of the ascending arrangement of all distinct 
elements of {a, b} U { (i- e )(7+7-i) : ^^}U {p+^^ppiy e (a, 6) : £ £ Z}. T/ien, 6ot/i <?(p) and 

h{p) are constants for any p £ (a,/3). 



Proof. Since a and /? are two consecutive elements of the ascending arrangement of all distinct 
elements of the set, it must be true that there is no integer I such that a < ^ P 

or a < 



(l-e)^+7-l) 

— < /3. It follows that there exist two integers I and £' such that (a, (5) C 



(l-e)(£+ 7 )' (l-e)(^+7-l) 

[TOl we have g{p) = g 



(l+e)(M- 7 -l) 

( (1-^+7) ' (i-skI+7-i) ) and (a '^ - ( d- E y+7) ' (i-^+7-i) )- ^P^g Lemma [9] and Lemma 

^ and h(p) = h ( (1 _ £ 7 K ^ +7) ) for any p € (a, 0). 

□ 



(l-e)(£+ 7 -l) 



Lemma 12 For any p G (0, 1), lim t i C(p + t) > C(p) and lim t | C(p — t)> C(p). 



Proof. Observing that g(p + t)< g{p) for any t > and that 



h(p + t) = h(p) + 



7-1 



(l-e)(p + t) 



7-1 ' 
(1 - e)p 



hip) 



for -1 < 2=1 

ior i < (1 _ E)(p+t) 



7-1 
(l-e)p 



< 0, i.e., < t < -tf-^ ( it l^_ _ _ Pi we have 



5(7,9(p + t),% + i),P + *) > S(i,g(p),h(p),p + t) 



(17) 



for < t < max <^ 1, ( 



7-1 
(l-e)p 



we have 



gip + t) 



g(p + t)= g(p) + 



g(p) - 1 for j^- 



- p. Since 

7-1 
(l + e)(p + i) 



7-1 
(l + e)p. 



'-1 



s(p) fOT (tt^; ^ 



7-1 
(1+£)P 

7-1 
(1+£)P 



0<<< -A 



< t < 



7-1 
(1+£)P 

7-1 
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It follows that both g{p + t) and h(p + t) are independent of t if t > is small enough. Since 
5(7,5, h,p + t) is continuous with respect to t for fixed 5 and /i, we have that lim^o S( , y,g(p + 
t), h(p + t),p + t) exists. As a result, 



]5mC(p + t) = Km S(i,g(p + t),h(p + t),p + t) 

- 1 X S '( 7 ' 5 ^)' /t ^)' p + *) = Sili dip), h(p),p) = C(p), 

where the inequality follows from Q17j) . 

Observing that /i(p — £) > fo(p) for any t > and that 



9 (p - t) = flf(p) + 



7-1 



(l + e)(p-t) 



7-1 
(l + e)p. 



g(p) 



for < i < p 



for < t < p 



7-1 



7-1 



1 + 



7-1 
(i+.-)p 



-1 



we have 



5(7, g(p - t), h(p - t),p - t) > S{j,g(p),h(p),p -t) 
1 



1 + 



7-1 

(l+e)p 



. Since 



(18) 



h{p -t)= h{p) + 



7-1 



(l- e )(p-t) 



7-1 ' 
(1 - e)p 



we have 



Hp - t) 



h(p) + l for^ 



e)p 

for 7^4- 

(l-e)p ' 



7-1 
(l-e)p 

7-1 
(l-e)P 



& < i < p 



& < t <p-^- 



7-1 

(1-6) 

7-1 



7-1 

(l-e)p 



7-1 
(l-e)p 
-1 



It follows that both g{p — t) and /i(p — t) are independent of t if t > is small enough. Since 
5(7,5, h,p — t) is continuous with respect to t for fixed 5 and h, we have that lim^o S(j,g(p — 
t), h(p — t),p — t) exists. Hence, 

hmC(p-t) = lim S (7, g(p-t),h(p-t),p-t) 

~ ^^'^'^'P ~ *) = S< ^i9{p)iHp)iP) = c(p)> 



□ 



where the inequality follows from (|18[) . 



Lemma 13 Lei < it < v < 1, h > and g < h. Then, 



min 5(7,5, hp) = min{5(7,5, h,u), S(j, g,h,v)}. 



25 



Proof. Since Pr{fc < k} = Ipipf, k + 1) where I p is the regularized incomplete beta function 



I p (a,b) 



B(p,a,b) _ f*1T- 1 (l-t) b - 1 dt 



we have 



B(a,b) 
d[Pr{fc < I}] 



> 



(19) 



dp B(7,Z + 1) 

for any integer Z > 0. To show the lemma, it suffices to consider 2 cases as follows. 

Case (i): g < < h. In this case, S(j,g, h,p) = -5(7,0, h,p), which is increasing as a result of 

USD- 
Case (ii): < g < h. By (QjJJ), for two integers < k < I, 

d[Pr{k <k<l}} p 7_1 (! - pY P 7_1 (! ~ P) k 



dp 



B(j,l + l) B(j,k + 1) 
pt^il -p) k r 



B(7,/ + l) 



(1-P) 



Z!(7 + fc)! 
fc!( 7 + 0! 



> 



if p < 1 — gfejpni • If follows that S(j, g, h,p) is a unimodal function of p. 

From such investigation of the derivative of C(p) = 5(7, g, h,p) with respective to p, we can see 
that one of the following three cases must be true: (1) C(p) decreases monotonically for p G [u, v]; 
(2) C{p) increases monotonically for p G [u, v]; (3) there exists a number 9 G (u, v) such that C(p) 
increases monotonically for p £ [u,6] and decreases monotonically for p E (8,v\. It follows that 
the lemma must be true for all cases. 

□ 



Lemma 14 Let a < (3 be two consecutive elements of the ascending arrangement of all distinct 
elements of {a,b} U {jjz^pp^ G (a, 6) : ^ G Z} U { (1+£) 7 ( ^ 7 _ 1} G (a, b) : t G Z}. Then, 
C(p) > min{C(a), C(/3)} for any p G (a,/3). 

Proof. By Lemma [TTj g(p) and h(p) are constants for any p G (a, ft). Hence, we can drop the 
argument p and write g{p) = 3, h(p) = h and C(p) = 5(7, g, h,p). 

For p G (a,f3), define interval [a + t,/3 — t] with < < < min fj> — a, j3 — p, J . Then, 
p G [a + t , [5 — t] . By Lemma [13j 

C(p) > min C(/i) = min{C(a + t), C(/3 - t)} 

/i6[a+t,/9-t] 

for < t < min (p — a,j3 — p, . By Lemma [T21 both lim t j_o C{a + t) and lim^ C{f3 — t) exist 
and are bounded from below by C(a) and C(f3) respectively. Hence, 

C{p) > lim min{C(a + t), C(fi-t)} 

= min|limC(a + i), \mxG{j3-t) \ > min{C*(a), C(/3)} 

I £J.0 £J.0 I 
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for any p G (a, /?). 



□ 



Finally, we can readily deduce Theorem 3. The first statement on the minimum of the coverage 
probability follows immediately from Lemma PT41 The second statement on the minimum of the 
coverage probability can be proved in a similar way. 



D The Incomplete Work of Dagum et al. 

Let Zi, Z2, ■ ■ ■ be a sequence of i.i.d. random variables defined on the same probability space 
such that Zi G [0, 1] and E[Zj] = pz G (0, 1). In order to estimate pz, Dagum et al. proposed (in 
Section 2.1, page 1486 of [B]) the following Stopping Rule Algorithm: 

Initialize N <- 0, S «- 0. 

While S < Ti do: N <- N + 1, S ^ S + Z N . 

Return pz = as the estimate of /^^. 

In Section 2.1, page 1486 of [6], Dagum et al. claimed that the reliability of the estimate pz 
is asserted by the following "Stopping Rule Theorem". 

Theorem 4 Let e G (0, 1) and 5 G (0, 1). T/ien, Pr{|/2 Z - < efi z } >1-S. 

We would like to point out that the proof of Dagum et al. is not complete. There exists a 
significant gap which cannot be patched by using their argument. 



D.l The Proof of "Stopping Rule Theorem" by Dagum et al 

To exhibit the fallacy of the argument by Dagum et al., we shall represent their proof for "Stopping 
Rule Theorem" in this section. The following preliminary result is first established by Dagum et 
al. as Lemma 4.6 in page 1489 of [6]. 

Lemma 15 Let A = e — 2. Let pz = max{<r|, epz} where a 2 z is the variance of Z . Define 
= Yli=i(Zi — Vz) for k = 1, 2, • • • . Then, for any fixed N > and any (3 G [0, 2\pz], 

pJf^Uexpf^ (20) 



N ~ ) ~ \ 4Xp z 

and 

(^7 '■ (21) 



The argument of Dagum et al. (in Section 5, page 1490-1491 of [B]) for the "Stopping Rule 
Theorem" proceeds as follows. Let Nz be the sample size at the stopping time. Recall that 
V-z = jyj' ^ suffices to show that 

Pr [n z < — ^-^\ < - (22) 
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and that (equation (8), page 1491 of [6]) 

Pr [n z > 



Ti 



(J>z(l - e) 



< 



(23) 



To show (|22p . it suffices to consider the case that /^z(l + e) < 1, since the theorem is trivially 



true if nzO- + e) > 1- Let L 



Mz(1+e)_ 

! + (! + £)# In 



By the definitions of Ti and L, 



> 



1 + (1 + £)M In 



4A , 2 

1 ^ "2— ln A' 

e Hz a 



(24) 



Since Nz is an integer, iV^ < 



Ti 



implies Nz < L. But iV^ < L if and only if Si > Tj. 



Thus, Pr{^ z < ^t^jj} < Pr{A^z < L} = Pr{S L > TJ where S L = £f =1 ^- Let /? = ^ - 
Then, Pr{S L > Ti} = Pr{S L - fji z L - pL > 0} = Pr {% > /?}. Noting that £/U Z < /3 < 2Ap z , 
Lemma [T51 implies that Pr{^ > p 1 } < exp (r^pj < cxp f— ix^~V Using the last inequality of 



24|) and noting that pz < max{/i^(l — fj, z ), ^Hz} < /^z, it follows that Pr jiVz < | < 

Pr {%>/?}< exp ( ~ L 4 ( A ^ z z)2 ) < f • This completes the proof of (l22|) . 

Finally, instead of giving detailed argument in [6], Dagum et al. claimed that the proof of 
(1231) is similar. 



D.2 A Hole in the Proof of Dagum et al 

We would like to point out that the proof of Dagum et al. is not complete because (I23p cannot 
be shown by a similar argument of Dagum et al. in proving (|22[) . The gap is exhibited as follows. 
To show Pr jiVz > Mz ^ f 1 1 _ £ - ) j < § in the similar spirit of the first part, it is expected to construct 
an integer L and a real number (3 = ^z — -77 such that 

N z > 7* r | C {5 L <Tx}, (25) 



Mz(l-e). 

epz < A (26) 

/3 < 2Apz, (27) 
4A , 2 
1 — ln 7 



L > 72— ^ 7 (28) 



and consequently 

Pr J /v, -> 

Mz(l - £ 



J^ 1 - < Pr{5z, < Ti} (29) 

= Pr|^<-/3| (30) 

s exp (~Tv7^~) <*> 

< { (34) 
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where (|29|) relies on ([25]) ; ([30]) is due to the definitions of [3 and Sl; ([31]) relies on ([21]) of Lemma 
[El and (|27|) : (i32|) relies on ([26]); ([33]) is due to the fact p z < max{fi z (l - Hz), e^z} < V>z\ (I341) 
relies on (|28|) . 

Unfortunately, it is possible that (|26|) contradicts (|25p ! To see this, note that, by the definition 



of (5 and 



we have epz < — 37, i-e., £ > 



since L is an integer. We can show 



that 



{ N z > ^fe} ^ {<Sz < T il is not true if i > 



Ti 
(l-e)M2 



For this purpose, note that 



> Mz (i-e) } = {-^-Z > = {^a- < Ti} where if = (1 _^ Mz j • For a random variable Z with 



mean value [iz such that 



is not an integer, we have K 



(l-e)fj, 2 



< 



< L. As a 



(l-£)^tZ 

result, it is possible that Zi <T\ < J2i=i Z%, which implies that {Sk < T 1} C {Sl < Ti} 



is not true. Thus we have shown that (|25p is not necessarily true if (|26p is satisfied. This 
demonstrates that it is not possible to show Pr jiVz > ^j^Tyj < | by using the argument of 
Dagum et al. 



E The Fallacy of Cheng's Reasoning 

In order to improve efficiency, Cheng revised the Stopping Rule Algorithm of Dagum et al. by 
replacing the threshold value Ti with a smaller number a. See, pages 12-13, Algorithm 1 of 
Section 5, and page 18, lines 9-10 of his paper [3J. 

Cheng claimed that such a revised algorithm ensures Prjl//^ — Hz\ < e Hz} > 1 — S. He first 
established Theorem 4 in page 7 of his paper, which is restated as Theorem [5] follows. 



Theorem 5 Let Z\,--- ,Z n be i.i.d. random variables bounded in [0, 1] with common mean value 
HZ € (0,1). Let < e < min{l, (1 - Hz)/vz}- Then, Pr{| E "= lZ ' - u z \ <eu z ] > 1 - 5 if 

11 - ^(l+e)ln(l+e)-e ln (I) ■ 

In the first paragraph of page 18 of his paper [JJ, after defining events 



Cheng applied the law of total probability to write 

4 

Pr{\Ji z -iiz\ <e^z} = Y, PT i\^-^z\ <efi z | ^}Pr{^} 

i=l 

and attempted to show that the right-hand side of the equality is bounded from below by 1 — S. 
Unfortunately, Cheng made a fundamental mistake in bounding Pr {\jlz — pz\ < z^z I E{\ and 
other terms alike. He noted that Sm z = Yl7=i-^-i — a anc ^ thus > ^7 i n (i+ e )-^/(i+ e ) ^ n \T~) 
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because of the definitions of the stopping rule and a. Conditioning upon nz > j+fj with some 
constant k, he obtained 



N z > 



1 1 + e 



In 



HZ 1 + ke (1 + e) ln(l + e) - e Hz (1 + e) hi(l + e) - e 
and then applied Theorem [5] to claim 



In — 



1 + E 
1 + fce 



(35) 



Pr 



- Hz 



(J>z 



<£ \ fJ-Z > 



HZ 

1 + ke 



> 1 



1 + e 
1 + k.E 



(36) 



which was equation (28) of page 17 in his paper [3]. 

Here Cheng made a subtle and critical mistake by illegally applying Theorem The reason 
is that the sample size requirement of Theorem [5| is independent of samples, while the validity of 
1135]) depends on samples. Consequently, ([36]) is not justified. This affects subsequent relevant 
development. 
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